Report acceleration using intermediate results in a distributed indexer system for searching events

ABSTRACT

A method and system for managing searches of a data set that is partitioned based on a plurality of events. A structure of a search query may be analyzed to determine if logical computational actions performed on the data set is reducible. Data in each partition is analyzed to determine if at least a portion of the data in the partition is reducible. In response to a subsequent or reoccurring search request, intermediate summaries of reducible data and reducible search computations may be aggregated for each partition. Next, a search result may be generated based on at least one of the aggregated intermediate summaries, the aggregated reducible search computations, and a query of adhoc non-reducible data arranged in at least one of the plurality of partitions for the data set.

CROSS-REFERENCE TO RELATED APPLICATION

This utility patent application is based upon U.S. Provisional PatentApplication No. 61/649,125 filed on May 18, 2012, the benefit of whichis hereby claimed under 35 U.S.C. §119(e) and incorporated herein byreference.

TECHNICAL FIELD

The invention is directed to providing services for use by a customer,and more particularly, to providing search services to the customer overa network.

BACKGROUND OF THE INVENTION

Many companies install and maintain search software for use on a varietyof distributed systems that can range from a single computer for a smallbusiness to a collection of servers and a plurality of user computernodes for a large corporation. In the past, search reports based onrelatively large sets of indexed data were time consuming to generate.To reduce latency in search report generation for a larger data setintermediate summaries for reoccurring search reports for partitioneddata have been periodically generated and stored in a separate index.Subsequently, a search report run on the data set would aggregate thecorresponding pre-computed intermediate summaries to generate the searchreport in a relatively shorter period of time. However, the nature ofdifferent types of data and the structure of different search querieshas made it difficult to manage and configure these intermediatesummaries. For example, some partitions may include data that is highlyreducible for a search query and other partitions may include data thatis only marginally reducible for the same search query. Also, for asingle set of data in a partition, it may be reducible for some queriesbut not others. Other difficulties include configuring the amount ofdata included in the partitions of the data set, such as adding newlyidentified data to a partition after intermediate summaries aregenerated. Consequently, systems that can manage search reportgeneration for relatively large sets of data are the subject ofconsiderable innovation.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments are described with referenceto the following drawings. In the drawings, like reference numeralsrefer to like parts throughout the various figures unless otherwisespecified.

For a better understanding, reference will be made to the followingDescription Of The Various Embodiments, which is to be read inassociation with the accompanying drawings, wherein:

FIG. 1 illustrates a system environment in which various embodiments maybe implemented;

FIG. 2A shows a schematic drawing of a rack of blade servers;

FIG. 2B illustrates a schematic embodiment of a blade server that may beincluded in a rack of blade servers such as that shown in FIG. 2A;

FIG. 3 shows a schematic embodiment of a mobile device;

FIG. 4 illustrates a schematic embodiment of a network device;

FIG. 5 shows a flowchart for general processing a data set and provide aresult in response to a search query;

FIG. 6 illustrates a flowchart for processing intermediate summaries andreducible search computations for partitioned data;

FIG. 7 shows a flowchart for processing a search request;

FIG. 8 illustrates a schematic diagram of a data set that includes aplurality of partitions;

FIG. 9 shows an exemplary table of data; and

FIG. 10 illustrates an exemplary table of intermediate summaries inaccordance with the various embodiments.

DESCRIPTION OF THE VARIOUS EMBODIMENTS

The invention now will be described more fully hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, specific embodiments by which theinvention may be practiced. This invention may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. Amongother things, the invention may be embodied as methods or devices.Accordingly, the invention may take the form of an entirely hardwareembodiment, an entirely software embodiment or an embodiment combiningsoftware and hardware aspects. The following detailed description is,therefore, not to be taken in a limiting sense.

Throughout the specification and claims, the following terms take themeanings explicitly associated herein, unless the context clearlydictates otherwise. The phrase “in one embodiment” as used herein doesnot necessarily refer to the same embodiment, though it may.Furthermore, the phrase “in another embodiment” as used herein does notnecessarily refer to a different embodiment, although it may. Thus, asdescribed below, various embodiments may be readily combined, withoutdeparting from the scope or spirit of the invention.

In addition, as used herein, the term “or” is an inclusive “or”operator, and is equivalent to the term “and/or,” unless the contextclearly dictates otherwise. The term “based on” is not exclusive andallows for being based on additional factors not described, unless thecontext clearly dictates otherwise. In addition, throughout thespecification, the meaning of “a,” “an,” and “the” include pluralreferences. The meaning of “in” includes “in” and “on.”

Various embodiments now will be described more fully hereinafter withreference to the accompanying drawings, which form apart hereof, andwhich show, by way of illustration, specific exemplary embodiments bywhich the invention may be practiced. The embodiments may, however, beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the embodiments to those skilled in the art.Among other things, the various embodiments may be methods, systems,media or devices. Accordingly, the various embodiments may take the formof an entirely hardware embodiment, an entirely software embodiment oran embodiment combining software and hardware aspects. The followingdetailed description is, therefore, not to be taken in a limiting sense.

For example embodiments, the following terms are also used hereinaccording to the corresponding meaning, unless the context clearlydictates otherwise.

The term “query” as used herein refers to commands and/or sequences ofcommands that are directed for searching and/or retrieving data from arecord datastore. Queries generally produce a result or results based onthe form and structure of the particular query. Query results may besorted and grouped based on the structure and form of the query. In atleast one of the various embodiments, queries may includes operators andfunctions for calculating value based on the stored records, includingfunctions that produce results sets that may include statistics andmetrics about the portions of the record datastore. Structure QueryLanguage (SQL) is a well know query language often used to form queriesfor relational databases; herein queries may be described using a“SQL-like” form for readability. However, the various embodiments arenot limited to using SQL-like formatting for queries and other well knowquery languages and/or custom query languages may be employed consistentwith what is claimed herein.

“Index”, “Indexer”, “Indexing”, and “Index Storage” as used herein mayrepresent elements of described embodiments that may index and storedata and events. Indexers may collect, parse, and store data tofacilitate fast and accurate information retrieval. Index design mayincorporate interdisciplinary concepts from linguistics, cognitivepsychology, mathematics, informatics, physics, and computer science.Also, indexes may reside in flat files in a data store on a file systemwhich may be distributed and enables distributed computations. Indexfiles may be managed to facilitate flexible searching and fast dataretrieval, eventually archiving them according to a configurableschedule. During indexing, incoming raw data may be processed to enablefast search and analysis, the results of which may be stored in anindex. As part of the indexing process, the indexer may add knowledge tothe data in various ways, including by: separating a data stream intoindividual, searchable events; creating or identifying timestamps;extracting fields such as host, source, and source type; performinguser-defined actions on the incoming data, such as identifying customfields, masking sensitive data, writing new or modified keys, applyingbreaking rules for multi-line events, filtering unwanted events, routingevents to specified indexes or servers, and the like. Also, for areducible structure of a reoccurring search, the indexer can perform alogical sequence of actions on the data, e.g., computing averages,variances, root mean squares or the like.

Briefly stated, various embodiments are directed towards a system andmethod for managing and configuring one or more different search reportsfor a data set that is partitioned based on a time of occurrence for aplurality of events. Data corresponding to each event may be arranged ina corresponding partition. In at least one of the various embodiments, astructure of a search query for a particular search report may beanalyzed to determine if at least a portion of the query's logicalcomputational actions are reducible. If so, the reducible searchcomputations are performed on at least one of the partitions. Also, inat least one of the various embodiments, data in each partition isanalyzed to determine if at least a portion of the data is reducible fora particular search report. If so, intermediate summaries are generatedfor the reducible data which may be arranged in one or more partitions.In response to a subsequent or reoccurring request for a particularsearch report, the corresponding intermediate summaries and reduciblesearch computations may be aggregated for each partition. Next, a searchresult may be generated based on at least one of the aggregatedintermediate summaries, the aggregated reducible search computations,and a search query of any remaining adhoc non-reducible data arranged inat least one of the plurality of partitions for the data set.Additionally, intermediate summaries and reducible search computationsmay be reused for more than one particular search report.

In at least one of the various embodiments, each partition of the dataset is based on at least a time for an event. Also, in at least one ofthe various embodiments, the data may include time series data whereeach data record has a timestamp associated with it. Search reports forthe stored time series data may be limited by a range of time (such asbetween one time and another, or data earlier than a given time and thelike) and/or provide search results ordered by a time (such as fromearliest-to-latest or vice versa). Additionally, in at least one of thevarious embodiments, newly identified other data related to an event maybe appended to a particular partition that corresponds to the event.

In at least one of the various embodiments, at least a portion of atleast one partition of the data set can include adhoc data that issubstantially unique/non-reducible for a particular search report. Forexample, in at least one of the various embodiments, if the amount ofadhoc non-reducible data in a partition for a particular search reportis substantially greater than reducible data, the process may stopgenerating further intermediate summaries for that partition for thatparticular search report unless more reducible data is appended to thepartition. In at least one of the various embodiments, reoccurrences ofthe search report may be one of periodic, intermittent, or random.

In at least one of the various embodiments, a logical sequence ofactions performed by a reducible search computation for a particularsearch report may be normalized for the partitioned data set.Additionally, in at least one of the various embodiments, a uniquenessvalue for at least a portion of the data in each partition for aparticular search report may be determined. Also, a high uniquenessvalue may be employed to identify the non-reducible data and a lowuniqueness value may be employed to identify reducible data in eachpartition for a particular search report. Additionally, the data may bestored in one or more data stores, and one or more computing devices maybe employed to distribute the computing of intermediate summaries,reducible search computations, and search results for a plurality ofdifferent search reports.

The following briefly describes embodiments in order to provide a basicunderstanding of some aspects of the invention. This brief descriptionis not intended as an extensive overview. It is not intended to identifykey or critical elements, or to delineate or otherwise narrow the scope.Its purpose is merely to present some concepts in a simplified form as aprelude to the more detailed description that is presented later.

Additionally, it should be appreciated that though many embodimentsdescribed herein may be cloud-based, embodiments should not beconsidered so limited. One of ordinary skill in the art will recognizethat enabling embodiments may be arranged to be partially deployed incloud-based configurations with some elements in the cloud and someelements separate from cloud based resources. Likewise, enablingembodiments may be arranged to be deployed and operate in configurationsentirely separate from cloud-based resources.

Illustrative Operating Environment

FIG. 1 shows components of an environment in which various embodimentsmay be practiced. Not all of the components may be required to practicethe various embodiments, and variations in the arrangement and type ofthe components may be made without departing from the spirit or scope ofthe various embodiments.

In at least one embodiment cloud network 102 enables one or more networkservices for a user based on the operation of corresponding arrangements104 and 106 of virtually any type of networked computing device. Asshown, the networked computing devices may include server network device112 host network device 114, enclosure of blade servers 110, enclosureof server computers 116, super computer network device 118, and thelike. Although not shown, one or more mobile devices may be included incloud network 102 in one or more arrangements to provide one or morenetwork services to a user. Also, these arrangements of networkedcomputing devices may or may not be mutually exclusive of each other.

Additionally, the user may employ a plurality of virtually any type ofwired or wireless networked computing devices to communicate with cloudnetwork 102 and access at least one of the network services enabled byone or more of arrangements 104 and 106. These networked computingdevices may include tablet mobile device 122, handheld mobile device124, wearable mobile device 126, desktop network device 120, and thelike. Although not shown, in various embodiments, the user may alsoemploy notebook computers, desktop computers, microprocessor-based orprogrammable consumer electronics, network appliances, mobiletelephones, smart telephones, pagers, radio frequency (RF) devices,infrared (IR) devices, Personal Digital Assistants (PDAs), televisions,integrated devices combining at least one of the preceding devices, andthe like.

One embodiment of a mobile device is described in more detail below inconjunction with FIG. 3. Generally, mobile devices may include virtuallyany substantially portable networked computing device capable ofcommunicating over a wired, wireless, or some combination of wired andwireless network.

In various embodiments, network 102 may employ virtually any form ofcommunication technology and topology. For example, network 102 caninclude local area networks Personal Area Networks (PANS), (LANs),Campus Area Networks (CANs), Metropolitan Area Networks (MANs) Wide AreaNetworks (WANs), direct communication connections, and the like, or anycombination thereof. On an interconnected set of LANs, including thosebased on differing architectures and protocols, a router acts as a linkbetween LANs, enabling messages to be sent from one to another. Inaddition, communication links within networks may include virtually anytype of link, e.g., twisted wire pair lines, optical fibers, open airlasers or coaxial cable, plain old telephone service (POTS), waveguides, acoustic, full or fractional dedicated digital communicationlines including T1, T2, T3, and T4, and/or other carrier and other wiredmedia and wireless media. These carrier mechanisms may includeE-carriers, Integrated Services Digital Networks (ISDNs), universalserial bus (USB) ports, Firewire ports, Thunderbolt ports, DigitalSubscriber Lines (DSLs), wireless links including satellite links, orother communications links known to those skilled in the art. Moreover,these communication links may further employ any of a variety of digitalsignaling technologies, including without limit, for example, DS-0,DS-1, DS-2, DS-3, DS-4, OC-3, OC-12, OC-48, or the like. Furthermore,remotely located computing devices could be remotely connected tonetworks via a modem and a temporary communication link. In essence,network 102 may include virtually any communication technology by whichinformation may travel between computing devices. Additionally, in thevarious embodiments, the communicated information may include virtuallyany kind of information including, but not limited to processor-readableinstructions, data structures, program modules, applications, raw data,control data, archived data, video data, voice data, image data, textdata, and the like.

Network 102 may be partially or entirely embodied by one or morewireless networks. A wireless network may include any of a variety ofwireless sub-networks that may further overlay stand-alone ad-hocnetworks, and the like. Such sub-networks may include mesh networks,Wireless LAN (WLAN) networks, Wireless Router (WR) mesh, cellularnetworks, pico networks, PANs, Open Air Laser networks, Microwavenetworks, and the like. Network 102 may further include an autonomoussystem of intermediate network devices such as terminals, gateways,routers, switches, firewalls, load balancers, and the like, which arecoupled to wired and/or wireless communication links. These autonomousdevices may be operable to move freely and randomly and organizethemselves arbitrarily, such that the topology of network 102 may changerapidly.

Network 102 may further employ a plurality of wired and wireless accesstechnologies, e.g., 2nd (2G), 3rd (3G), 4th (4G), 5^(th) (5G) generationwireless access technologies, and the like, for mobile devices. Thesewired and wireless access technologies may also include Global Systemfor Mobile communication (GSM), General Packet Radio Services (GPRS),Enhanced Data GSM Environment (EDGE), Code Division Multiple Access(CDMA), Wideband Code Division Multiple Access (WCDMA), Long TermEvolution Advanced (LTE), Universal Mobile Telecommunications System(UMTS), Orthogonal frequency-division multiplexing (OFDM), Wideband CodeDivision Multiple Access (W-CDMA), Code Division Multiple Access 2000(CDMA2000), Evolution-Data Optimized (EV-DO), High-Speed Downlink PacketAccess (HSDPA), IEEE 802.16 Worldwide Interoperability for MicrowaveAccess (WiMax), ultra wide band (UWB), user datagram protocol (UDP),transmission control protocol/Internet protocol (TCP/IP), any portion ofthe Open Systems Interconnection (OSI) model protocols, Short MessageService (SMS), Multimedia Messaging Service (MMS), Web Access Protocol(WAP), Session Initiation Protocol/Real-time Transport Protocol(SIP/RTP), or any of a variety of other wireless or wired communicationprotocols. In one non-limiting example, network 102 may enable a mobiledevice to wirelessly access a network service through a combination ofseveral radio network access technologies such as GSM, EDGE, SMS, HSDPA,LTE and the like.

Enclosure of Blade Servers

FIG. 2A shows one embodiment of an enclosure of blade servers 200, whichare also illustrated in FIG. 1. Enclosure of blade servers 200 mayinclude many more or fewer components than those shown in FIG. 2A.However, the components shown are sufficient to disclose an illustrativeembodiment. Generally, a blade server is a stripped down servercomputing device with a modular design optimized to minimize the use ofphysical space and energy. A blade enclosure can include several bladeservers and provide each with power, cooling, network interfaces,input/output interfaces, and resource management. Although not shown, anenclosure of server computers typically includes several computers thatmerely require a network connection and a power cord connection tooperate. Each server computer often includes redundant components forpower and interfaces.

As shown in the figure, enclosure 200 contains power supply 204, andinput/output interface 206, rack logic 208, several blade servers 210,212, 214, and 216, and backplane 202. Power supply 204 provides power toeach component and blade server within the enclosure. The input/outputinterface 206 provides internal and external communication forcomponents and blade servers within the enclosure. Backplane 208 canenable passive and active communication of power, logic, input signals,and output signals for each blade server.

Illustrative Blade Server

FIG. 2 a illustrates an illustrative embodiment of blade server 250,which may include many more or fewer components than those shown. Asshown in FIG. 2A, a plurality of blade servers may be included in oneenclosure that shares resources provided by the enclosure to reducesize, power, and cost.

Blade server 250 includes processor 252 which communicates with memory256 via bus 254. Blade server 250 also includes input/output interface290, processor-readable stationary storage device 292, andprocessor-readable removable storage device 294. Input/output interface290 can enable blade server 250 to communicate with other blade servers,mobile devices, network devices, and the like. Interface 190 may providewireless and/or wired communication links for blade server.Processor-readable stationary storage device 292 may include devicessuch as an electromagnetic storage device (hard disk), solid state harddisk (SSD), hybrid of both an SSD and a hard disk, and the like. Also,processor-readable removable storage device 291 enables processor 252 toread non-transitive storage media for storing and accessingprocessor-readable instructions, modules, data structures, and otherforms of data. The non-transitive storage media may include Flashdrives, tape media, floppy media, and the like.

Memory 256 may include Random Access Memory (RAM), Read-Only Memory(ROM), hybrid of RAM and ROM, and the like. As shown, memory 256includes operating system 258 and basic input/output system (BIOS) 260for enabling the operation of blade server 250. In various embodiments,a general-purpose operating system may be employed such as a version ofUNIX, or LINUX™, or a specialized server operating system such asMicrosoft's Windows Servers™ and Apple Computer's IoS Server™.

Memory 256 further includes one or more data storage 270, which can beutilized by blade server 250 to store, among other things, applications280 and/or other data. Data stores 270 may include program code, data,algorithms, and the like, for use by processor 252 to execute andperform actions. In one embodiment, at least some of data store 270might also be stored on another component of blade server 250,including, but not limited to, processor-readable removable storagedevice 294, processor-readable stationary storage device 292, or anyother processor-readable storage device (not shown). Data storage 270may include, for example, raw data 272, indexed data 274, andintermediate summaries 276.

Applications 280 may include processor executable instructions which,when executed by blade server 250, transmit, receive, and/or otherwiseprocess messages, audio, video, and enable communication with othernetworked computing devices. Examples of application programs includedatabase servers, file servers, calendars, transcoders, and so forth.Applications 280 may include, for example, search application 282, andindexing application 284.

Human interface components (not pictured), may be remotely associatedwith blade server 250, which can enable remote input to and/or outputfrom blade server 250. For example, information to a display or from akeyboard can be routed through the input/output interface 290 toappropriate peripheral human interface components that are remotelylocated. Examples of peripheral human interface components include, butare not limited to, an audio interface, a display, keypad, pointingdevice, touch interface, and the like.

Illustrative Mobile Device

FIG. 3 shows one embodiment of mobile device 300 that may include manymore or less components than those shown. Mobile device 300 mayrepresent, for example, at least one embodiment of mobile devices shownin FIG. 1.

Mobile device 300 includes processor 302 in communication with memory304 via bus 328. Mobile device 300 also includes power supply 330,network interface 332, audio interface 356, display 350, keypad 352,illuminator 354, video interface 342, input/output interface 338, hapticinterface 364, global positioning systems (UPS) receiver 358, Open airgesture interface 360, temperature interface 362, camera(s) 340,projector 346, pointing device interface 366, processor-readablestationary storage device 334, and processor-readable removable storagedevice 336. Power supply 330 provides power to mobile device 300. Arechargeable or non-rechargeable battery may be used to provide power.The power may also be provided by an external power source, such as anAC adapter or a powered docking cradle that supplements and/or rechargesthe battery. And in one embodiment, although not shown, a gyroscope maybe employed within mobile device 300 to measuring and/or maintaining anorientation of mobile device 300.

Mobile device 300 may optionally communicate with a base station (notshown), or directly with another computing device. Network interface 332includes circuitry for coupling mobile device 300 to one or morenetworks, and is constructed for use with one or more communicationprotocols and technologies including, but not limited to, protocols andtechnologies that implement any portion of the Open SystemsInterconnection (OSI) model for mobile communication (GSM), codedivision multiple access (CDMA), time division multiple access (TDMA),user datagram protocol (UDP), transmission control protocol/internetprotocol (TCP/IP), Short Message Service (SMS), Multimedia MessagingService (MMS), general packet radio service (GPRS), Web Access Protocol(WAP), ultra wide band (UWB), IEEE 802.16 Worldwide Interoperability forMicrowave Access (WiMax), Session Initiation Protocol/Real-timeTransport Protocol (SIP/RTP), General Packet Radio Services (GPRS),Enhanced Data GSM Environment (EDGE), Wideband Code Division MultipleAccess (WCDMA), Long Term Evolution Advanced (LTE), UniversalTelecommunications System (UMTS), Orthogonal frequency-divisionmultiplexing (OFDM), Code Division Multiple Access 2000 (CDMA2000)Evolution-Data Optimized (EV-DO), High-Speed Downlink Packet Access(HSDPA), or any of a variety of other wireless communication protocols.Network interface 332 is sometimes known as a transceiver, transceivingdevice, or network interface card (NIC).

Audio interface 356 is arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 356 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others and/or generate an audio acknowledgementfor some action. A microphone in audio interface 356 can also be usedfor input to or control of mobile device 300, e.g. using voicerecognition, detecting touch based on sound, and the like.

Display 350 may be a liquid crystal display (LCD), gas plasma,electronic ink, light emitting diode (LED), Organic LED (OLED) or anyother type of light reflective or light transmissive display that can beused with a computing device. Display 350 may also include a touchinterface 344 arranged to receive input from an object such as a stylusor a digit from a human hand, and may use resistive, capacitive, surfaceacoustic wave (SAW), infrared, radar, or other technologies to sensetouch and/or gestures. Projector 346 may be a remote handheld projectoror an integrated projector that is capable of projecting an image on aremote wall, or any other reflective object such as a remote screen.

Video interface 342 may be arranged to capture video images, such as astill photo, a video segment, an infrared video, or the like. Forexample, video interface 342 may be coupled to a digital video camera, aweb-camera, or the like. Video interface 342 may comprise a lens, animage sensor, and other electronics. Image sensors may include acomplementary metal-oxide-semiconductor (CMOS) integrated circuit,charge-coupled device (CCD), or any other integrated circuit for sensinglight.

Keypad 352 may comprise any input device arranged to receive input froma user. For example, keypad 352 may include a push button numeric dial,or a keyboard. Keypad 352 may also include command buttons that areassociated with selecting and sending images. Illuminator 354 mayprovide a status indication, and/or provide light. Illuminator 354 mayremain active for specific periods of time or in response to events. Forexample, when illuminator 354 is active, it may backlight the buttons onkeypad 352 and stay on while the mobile device is powered. Also,illuminator 354 may backlight these buttons in various patterns whenparticular actions are performed, such as dialing another mobile device.Illuminator 354 may also cause light sources positioned within atransparent or translucent case of the mobile device to illuminate inresponse to actions.

Mobile device 300 also comprises input/output interface 338 forcommunicating with external peripheral devices or other computingdevices such as other mobile devices and network devices. The peripheraldevices may include an audio headset, display screen glasses, remotespeaker system, remote speaker and microphone system, and the like.Input/output interface 338 can utilize one or more technologies, such asUniversal Serial Bus (USB), Infrared, WiFi, WiMax, Bluetooth™, and thelike. Haptic interface 364 is arranged to provide tactile feedback to auser of the mobile device. For example, the haptic interface 364 may beemployed to vibrate mobile device 300 in a particular way when anotheruser of a computing device is calling. Temperature interface 362 may beused to provide a temperature measurement input and/or a temperaturechanging output to a user of mobile device 300. Open air gestureinterface 360 may sense physical gestures of a user of mobile device300, for example, by using single or stereo video cameras, radar, agyroscopic sensor inside a device held or worn by the user, or the like.Camera 340 may be used to track physical eye movements of a user ofmobile device 300.

UPS transceiver 358 can determine the physical coordinates of mobiledevice 300 on the surface of the Earth, which typically outputs alocation as latitude and longitude values. GPS transceiver 358 can alsoemploy other geo-positioning mechanisms, including, but not limited to,triangulation, assisted GPS (AGPS), Enhanced Observed Time Difference(E-OTD), Cell Identifier (CI), Service Area Identifier (SAI), EnhancedTiming Advance (ETA), Base Station Subsystem (BSS), or the like, tofurther determine the physical location of mobile device 300 on thesurface of the Earth. It is understood that under different conditions,GPS transceiver 358 can determine a physical location for mobile device300. In at least one embodiment, however, mobile device 300 may, throughether components, provide other information that may be employed todetermine a physical location of the device, including for example, aMedia Access Control (MAC) address, IP address, and the like.

Human interface components can be peripheral devices that are physicallyseparate from mobile device 300, allowing for remote input and/or outputto mobile device 300. For example, information routed as described herethrough human interface components such as display 350 or keyboard 352can instead be routed through network interface 332 to appropriate humaninterface components located remotely. Examples of human interfaceperipheral components that may be remote include, but are not limitedto, audio devices, pointing devices, keypads, displays, cameras,projectors, and the like. These peripheral components may communicateover a Pico Network such as Bluetooth™, Zigbee™ and the like. Onenon-limiting example of a mobile device with such peripheral humaninterface components is a wearable computing device, which might includea remote pico projector along with one or more cameras that remotelycommunicate with a separately located mobile device to sense a user'sgestures toward portions of an image projected by the pico projectoronto a reflected surface such as a wall or the user's hand.

A mobile device may include a browser application that is configured toreceive and to send web pages, web-based messages, graphics, text,multimedia, and the like. The mobile device's browser application mayemploy virtually any programming language, including a wirelessapplication protocol messages (WAP), and the like. In at least oneembodiment, the browser application is enabled to employ Handheld DeviceMarkup Language (HDML), Wireless Markup Language (WML), WMLScript,JavaScript, Standard Generalized Markup Language (SGML), HyperTextMarkup Language (HTML), eXtensible Markup Language (XML), HTML5, and thelike.

Memory 304 may include Random Access Memory (RAM), Read-Only Memory(ROM), and/or other types of memory. Memory 304 illustrates an exampleof computer-readable storage media (devices) for storage of informationsuch as computer-readable instructions, data structures, program modulesor other data. Memory 304 stores a basic input/output system (BIOS) 308for controlling low-level operation of mobile device 300. The memoryalso stores an operating system 306 for controlling the operation ofmobile device 300. It will be appreciated that this component mayinclude a general-purpose operating system such as a version of UNIX, orLINUX™, or a specialized mobile computer communication operating systemsuch as Windows Mobile™ or the Symbian® operating system. The operatingsystem may include, or interface with a Java virtual machine modulo thatenables control of hardware components and/or operating systemoperations via Java application programs.

Memory 304 further includes one or more data storage 310, which can beutilized by mobile device 300 to store, among other things, applications320 and/or other data. For example, data storage 310 may also beemployed to store information that describes various capabilities ofmobile device 300. The information may then be provided to anotherdevice based on any of a variety of events, including being sent as partof a header during a communication, sent upon request, or the like. Datastorage 310 may also be employed to store social networking informationincluding address books, buddy lists, aliases, user profile information,or the like. Data storage 310 may further include program code, data,algorithms, and the like, for use by a processor, such as processor 302to execute and perform actions. In one embodiment, at least some of datastorage 310 might also be stored on another component of mobile device300, including, but not limited to, non-transitory processor-readableremovable storage device 336, processor-readable stationary storagedevice 334, or even external to the mobile device. Data storage 310 mayinclude, for example, raw data 312, index data 314 and intermediatesummaries 316.

Applications 320 may include computer executable instructions which,when executed by mobile device 300, transmit, receive, and/or otherwiseprocess instructions and data. Applications 320 may include, forexample, Search application 322, and Indexing Application 324. Otherexamples of application programs include calendars, search programs,email client applications, IM applications SMS applications Voice OverInternet Protocol (VOID) applications, contact managers, task managers,transcoders, database programs, word processing, programs, securityapplications, spreadsheet programs, games, search programs, and soforth.

Illustrative Network Device

FIG. 4 shows one embodiment of network device 400 that may be includedin a system implementing the invention. Network device 400 may includemany more or less components than those shown in FIG. 4. However, thecomponents shown are sufficient to disclose an illustrative embodimentfor practicing the present invention. Network device 400 may represent,for example, one embodiment of at least one of network device 112, 114or 120 of FIG. 1.

As shown in the figure, network device 400 includes a processor 402 incommunication with a memory 404 via a bus 428. Network device 400 alsoincludes a power supply 430, network interface 432, audio interface 456,display 450, keyboard 452, input/output interface 438,processor-readable stationary storage device 434, and processor-readableremovable storage device 436. Power supply 430 provides power to networkdevice 400.

Network interface 432 includes circuitry for coupling network device 400to one or more networks, and is constructed for use with one or morecommunication protocols and technologies including, but not limitedprotocols and technologies that implement any portion of the OpenSystems Interconnection model (OSI model), global system for mobilecommunication (GSM), code division multiple access (COMA), time divisionmultiple access (TDMA), user datagram protocol (UDP), transmissioncontrol protocol/Internet protocol (TCP/IP), Short Message Service(SMS), Multimedia Messaging Service (MMS), general packet radio service(GPRS), WAP, ultra wide band (UWB), IEEE 802.16 Worldwideinteroperability for Microwave Access (WiMax), Session InitiationProtocol/Real-time Transport Protocol (SIP/RTP), or any of a variety ofother wired and wireless communication protocols. Network interface 432is sometimes known as a transceiver, transceiving device, or networkinterface card (NIC). Network device 400 may optionally communicate witha base station (not shown), or directly with another computing device.

Audio interface 456 is arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 456 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others and/or generate an audio acknowledgementfor some action. A microphone in audio interface 456 can also be usedfor input to or control of network device 400, for example, using voicerecognition.

Display 450 may be a liquid crystal display (LCD), gas plasma,electronic ink, light emitting diode (LED), Organic LED (OLED) or anyother type of light reflective or light transmissive display that can beused with a computing device. Display 450 may be a handheld projector orpico projector capable of projecting an image on a wall or other object.

Network device 400 also may also comprise input/output interface 438 forcommunicating with external devices not shown in FIG. 4. Input/outputinterface 438 can utilize one or more wired or wireless communicationtechnologies, such as USB™, Firewire™, WiFi, WiMax, Thunderbolt™,Infrared, Bluetooth™, Zigbee™, serial port, parallel port, and the like.

Human interface components can be physically separate from networkdevice 400, allowing for remote input and/or output to network device400. For example, information routed as described here through humaninterface components such as display 450 or keyboard 452 can instead berouted through the network interface 432 to appropriate human interfacecomponents located elsewhere on the network. Human interface componentsinclude any component that allows the computer to take input from, orsend output to, a human user of a computer.

Memory 404 may include Random Access Memory (RAM), Read-Only Memory(ROM), and/or other types of memory. Memory 404 illustrates an exampleof computer-readable storage media (devices) for storage of informationsuch as computer-readable instructions, data structures, program modulesor other data. Memory 404 stores a basic input/output system (BIOS) 408for controlling low-level operation of network device 400. The memoryalso stores an operating system 406 for controlling the operation ofnetwork device 400. It will be appreciated that this component mayinclude a general-purpose operating system such as a version of UNIX, orLINUX™, or a specialized operating system such as MicrosoftCorporation's Windows® operating system, or the Apple Corporation's IOs®operating system. The operating system may include, or interface with aJava virtual machine module that enables control of hardware componentsand/or operating system operations via Java application programs.

Memory 404 further includes one or more data storage 410, which can beutilized by network device 400 to store, among other things,applications 420 and/or other data. For example, data storage 410 mayalso be employed to store information that describes variouscapabilities of network device 400. The information may then be providedto another device based on any of a variety of events, including beingsent as part of a header during a communication, sent upon request, orthe like. Data storage 410 may also be employed to store socialnetworking information including address books, buddy lists, aliases,user profile information, or the like. Data stores 410 may furtherinclude program code, data, algorithms, and the like, for use by aprocessor such as processor 402 to execute and perform actions. In oneembodiment, at least some of data store 410 might also be stored onanother component of network device 400, including, but not limited to,non-transitory media inside processor-readable removable storage device436 processor-readable stationary storage device 434, or any othercomputer-readable storage device within network device 400, or evenexternal to network device 400. Data storage 410 may include, forexample, raw data 412, indexed data 414, and intermediate summaries 416.

Applications 420 may include computer executable instructions which,when executed by network device 400, transmit, receive, and/or otherwiseprocess messages (e.g., SMS, Multimedia Messaging Service (MMS), InstantMessage (IM), email, and/or other messages) audio, video, and enabletelecommunication with another user of another mobile device. Otherexamples of application programs include calendars, search programs,email client applications, IM applications, SMS applications, Voice OverInternet Protocol (VOIP) applications, contact managers, task managers,transcoders, database programs, word processing programs, securityapplications, spreadsheet programs, games, search programs, and soforth. Applications 420 may include, for example, search application422, and indexing application 424.

General Operation

FIG. 5, illustrates a flowchart for general overview 500 of at least oneof the various embodiments for processing a search query for aparticular search report. Moving from a start block, the process stepsto block 502 where a data set is arranged in a plurality of partitionsbased at least it part on separate events over time.

Stepping to block 504, reducible search computations for the particularsearch report are performed for data at least one of the partitions. Asearch computation is a logical sequence of actions performed on data togenerate a result. A reducible search computation might generate anaverage, a variance, a root mean square, or the like that canpotentially be reused in one or more different search reports. Incontrast, a non-reducible search computation might instead requireaccessing most if not of the data in each partition each time aparticular search report is requested. For example, the non-reduciblesearch computation might involve computing a statistic that is groupedby the value of a high cardinality field, such as a user ID or an IPaddress. Since this type of computation is unlikely to be significantlyreducible, the intermediate summaries would be on par with the size ofthe original data. In this case, the process would skip generating theintermediate summaries for this partition of the data or that particularsearch report. In at least one of the various embodiments, at least someof the reducible search computations may be generated by an applicationthat also enables indexing of the data into one or more partitions thatcorrespond to events over time.

Advancing to block 506, one or more intermediate summaries are generatedfor reducible data for the particular search report in each partition.Also, depending on the nature of the data in each partition, a partitionmay include one or more intermediate summaries, adhoc data, or acombination of both intermediate summaries and adhoc data. In at leastone of the various embodiments, at least some of intermediate summariesare generated by an application that also enables indexing of the datainto one or more partitions that correspond to events over time.

At block 508, in response to a search query for a particular searchreport, the intermediate summaries and the reducible search computationsthat correspond to the search report are aggregated for the data set.

Then, at block 510, a search result is generated for the particularsearch report based at least one of the aggregated intermediatesummaries, aggregated reducible search computations, and the processingof any remaining adhoc data in one or more partitions with the searchquery. Next, the process returns to performing other actions.

FIG. 6 illustrates overview flowchart 600 for processing intermediatesummaries and reducible search computations for a particular searchreport for data that is arranged in a plurality of partitions. Movingfrom a start-block, the process advances to block 602 where the data setis arranged in a plurality of partitions that correspond to a time ofoccurrence for events. At decision block 604, a determination is made asto whether the structure of a particular requested search report isreducible. If negative, the logic loops back to block 602. However, ifthe determination at block 604 is affirmative, the process flows toblock 606 where reducible search computations are performed on data ineach partition for the particular search report. The values generated bythe reducible search computations are normalized for the data set.

At decision block 608, a determination is made as to whether reducibledata is present in a partition for a particular search report. If true,the process advances to block 610, where the process generatesintermediate summaries for reducible data in each partition for theparticular search report. In some cases, a very small portion of data ineach partition may be reducible, and/or the reduction in the data maynot be significant for a particular search report. Consequently, eventhough some intermediate summaries may be generated, the process maydetermine to not continue to do so if the advantage (reduction in datato be processed for a particular search report) is relatively small.

Moving to block 612, newly identified data for an event may be appendedto other data already arranged in a partition that corresponds to theevent. Also, if the determination at decision block 608 was false (noreducible data in at least one partition for a particular searchreport), the process would jump to block 612 and perform substantiallythe same actions as discussed above.

At decision block 614 if a period of time since the last processing ofthe partitioned data set has not yet elapsed, the process loops untiltrue and then loops back to block 602 where it performs substantiallythe same actions as discussed above.

FIG. 7 shows a flowchart of a general overview 700 for processing arequest for a particular search report. Moving from a start block, theprocess advances to block 702 where a query for a particular searchreport is received.

At block 704, the available intermediate summaries for the particularsearch report from one or more partitions are aggregated. Similarly, atblock 706, the available normalized reducible search computations forthe particular search report from one or more partitions are aggregated.

Moving to block 708, the search query for the particular search reportis performed on adhoc data that is non-reducible. Advancing to block710, a result for the particular search report is generated based atleast in part on the aggregated intermediate summaries, aggregatedreducible search computations, and the query performed on the adhocdata. Next, the process returns to performing other actions.

FIG. 8 illustrates a schematic diagram of data set 800 that includes aplurality of data partitions 802, 804, 806, 808, 810, and 812 thatcorrespond to events over time. As shown, partitions 802, 808 and 812include non-reducible adhoc data for a particular search report.Partition 804 includes 100 percent intermediate summaries for theparticular search report. Partitions 806 and 810 include partialintermediate summaries and partial adhoc data for the particular searchreport. Also, appended to partition 808 is newly identified data for theevent that corresponds to the partition.

FIG. 9 shows an exemplary table of raw data that includes columns fortime, name, size, and ip address on a server. Also, FIG. 10 illustratesan exemplary table of intermediate summaries that may be generated for aquery that needs the average and maximum of the size field overtimesplit by name. In addition to time and name, the table includes thecount of the size field, the maximum of the size field, and the sum ofthe size field for each value of time and name. This may be because inat least one of the various embodiments a separate accounting of thecount and the sum may be maintained to compute an accurate overallaverage (arithmetic mean).

It will be understood that figures, and combinations of actions in theflowchart-like illustrations, can be implemented by computer programinstructions. These program instructions may be provided to a processorto produce a machine, such that the instructions executing on theprocessor create a means for implementing the actions specified in theflowchart blocks. The computer program instructions may be executed by aprocessor to cause a series of operational actions to be performed bythe processor to produce a computer implemented process for implementingthe actions specified in the flowchart block or blocks. These programinstructions may be stored on some type of machine readable storagemedia, such as processor readable non-transitive storage media, or thelike.

Furthermore, it will be understood that for at least one of the variousembodiments, various types of data may be received and processed asdescribed and claimed herein. And, at least one of the variousembodiments is not limited to processing machine data.

1-20. (canceled)
 21. A computer-implemented method, comprising:receiving data on a computing device; generating a set of events,wherein each event in the set of events includes a portion of the data;associating a time with each event in the set of events; storing the setof events in a partition to which an indexer in a set of distributedindexers has access; calculating an intermediate result on dataextracted from the set of events in the partition, wherein the timesassociated with the set of events in the partition are included in botha first time period and a second time period that only partiallyoverlaps with the first time period; producing a first report coveringthe first time period by combining a partial result produced for thefirst time period by the indexer that has access to the partition with apartial result produced for the first time period by at least one otherindexer, wherein the partial result for the first time period that isproduced by the indexer with access to the partition is determined usingthe intermediate result; and producing a second report covering thesecond time period by combining a partial result produced for the secondtime period by the indexer with access to the partition with a partialresult produced for the second time period by at least one otherindexer, wherein the partial result for the second time period that isproduced by the indexer with access to the partition is determined usingthe intermediate result and without further using the set of events usedto calculate the intermediate result.
 22. The method of claim 21,wherein the partition is selected for storing the set of events based onthe times associated with the events in the set of events.
 23. Themethod of claim 21, wherein the partial result produced for the firsttime period by the indexer that has access to the partition and thepartial result produced for the first time period by the at least oneother indexer are generated concurrently.
 24. The method of claim 21,wherein an end of the second time period is later than an end of thefirst time period, and wherein the second report is produced after thefirst report is produced.
 25. The method of claim 21, wherein the firstreport and the second report are generated based on a same query. 26.The method of claim 21, wherein the intermediate result is calculatedbased on determining that a calculation needed to produce the firstreport and the second report is reducible.
 27. The method of claim 21,wherein the intermediate result is calculated based on determining thatdata in the set of events in the partition is reducible for purposes ofgenerating both the first report and the second report.
 28. The methodof claim 21, wherein the intermediate result is calculated based ondetermining that data in the set of events in the partition can besummarized for purposes of generating both the first report and thesecond report.
 29. The method of claim 21, wherein storing the set ofevents in the partition to which the indexer in the set of distributedindexers has access includes selecting the partition based on the timeassociated with the events in the set of events.
 30. The method of claim21, wherein the partial result for the second time period that isproduced by the indexer with access to the partition is determined usingboth the intermediate result and using data extracted from events in thepartition that were not used to calculate the intermediate result.
 31. Anon-transitory computer-readable medium having computer-executableinstructions for performing the method of claim
 21. 32. A system withone or more processors adapted to perform the method of claim 21.